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Abstract 


The present work stemmed from the study of the problem of harmonic analysis 
on the infinite-dimensional unitary group U{oo). That problem consisted in the 
decomposition of a certain 4-parameter family of unitary representations, which 
replace the nonexisting two-sided regular representation (Olshanski, J. Funct. Anal., 
2003). The required decomposition is governed by certain probability measures on 
an infinite-dimensional space fl, which is a dual object to U{oo). A way to describe 
those measures is to convert them into determinantal point processes on the real 
line; it turned out that their correlation kernels are computable in explicit form — 
they admit a closed expression in terms of the Gauss hypergeometric function 2 T 1 
(Borodin and Olshanski, Ann. Math., 2005). 

In the present work we describe a (nonevident) q'-discretization of the whole con¬ 
struction. This leads us to a new family of determinantal point processes. We reveal 
its connection with an exotic finite system of q-discrete orthogonal polynomials - 
the so-called pseudo big q-Jacobi polynomials. The new point processes live on a 
double q-lattice and we show that their correlation kernels are expressed through 
the basic hypergeometric function 2 ^ 1 - 

A crucial novel ingredient of our approach is an extended version G of 
the Gelfand-Tsetlin graph (the conventional graph describes the Gelfand-Tsetlin 
branching rule for irreducible representations of unitary groups). We find the q- 
boundary of G, thus extending previously known results (Gorin, Adv. Math., 2012). 


1 Introduction 

1.1 Origin of the problem 

By the infinite-dimensional unitary group we mean the inductive limit group U{oo) : = 
\^U{N). This group (as well as any its topological completion) is not locally compact and 
hence does not admit an invariant measure which is a necessary element for constructing 
the regular or biregular representation. Nevertheless, there exists a 4-parameter family 
of unitary representations of U{oo) x U ( 00 ) which are reasonable analogs of the biregular 
representations of the pre-limit groups U{N) x U{N) (Olshanski |3T]). The problem of 
harmonic analysis for U{oo) in the formulation of [HT] consists in the decomposition of 
these “generalized biregular representations” of U{oo) x U{oo) into a continuous integral 
of irreduciblle representations. 

As explained in Ell. for nonexceptional values of the parameters, the decomposition 
problem can be reduced to the description of a certain family {M.} of probability measures 
which live on an inhnite-dimensional space 

The space initially dehned as a kind of a dual object to the group U{oo), can 
be identihed with the boundary of the Gelfand-Tsetlin graph — a graded graph whose 
vertices of level = 1, 2,... represent the irreducible characters of U{N) and the graph 
structure reflects the branching rule of characters. 

Each measure A4 from our family is approximated by a canonical sequence 
M-i, M. 2 ^ ■ ■ ■ of discrete probability measures, which are dehned on the growing levels 
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of the Gelfand-Tsetlin graph. We call the latter measures the zw-measures, and the limit 
measures on G are called the boundary zw-measures. 

In Borodin-Olshanski [8] it is shown that every boundary zw-measure Ai can be turned 
into (the law of) a determinantal point process on the real line with two punctures, and 
the correlation kernel of that process can be explicitly computed: it is expressed in terms 
of the Gauss hypergeometric function 2 Fi. At present, this is the only way to describe 
the boundary zw-measures M.. 

The zw-measures Ad tv on the levels of the Gelfand-Tsetlin graph play a fundamental 
role in the whole theory, because the only existing way to handle the boundary measures 
M. relies on the approximation Ad tv Ad. 

There is a remarkable similarity between the zw-measures and /3 = 2 log-gas-type parti¬ 
cle systems (Forrester [TB]) arising in random matrix theory. Here f3 is Dyson’s parameter, 
which is similar to the Jack parameter in the theory of symmetric functions (Macdonald 
I2S1). Like log-gas systems, the zw-measures admit a deformation corresponding to ar¬ 
bitrary positive values of f3 (Olshanski [32]). Is it possible to go further and construct 
other deformations of the zw-measures, parallel to Macdonald’s {q, t)-deformation of the 
canonical scalar product in the algebra of symmetric functions? In the present work, we 
get an affirmative answer for the simplest non-Jack case, namely, for q = t. 

At hrst we believed that the desired g-analog of the zw-measures can be constructed 
with the use of the g-boundary of the Gelfand-Tselin graph — a concept, which was in¬ 
troduced in Gorin ira and then successfully exploited in Borodin-Gorin [6|. And indeed, 
we were able to do it for certain special values of the parameters corresponding to de¬ 
generate versions of the measures. However, in the case of general parameters, all our 
attempts failed. Eventually we realized what was the cause of failure. It turned out that, 
for our purposes, the very notion of the Gelfand-Tsetlin graph is not suitable and has to 
be extended. 

The key idea of the dehnition of the extended Gelfand-Tsetlin graph is the following: 
the vertices of the ordinary Gelfand-Tsetlin graph can be identihed with the hnite point 
conhgurations on the one-dimensional lattice Z C M, which label irreducible representa¬ 
tions of unitary groups. In the extended graph, the lattice Z has to be replaced by the 
double lattice Z U Z, which we identify with the double g-lattice of the form 

£ ;= {(_q^ : m e Z} U {C+g” : n G Z} C M \ {0}, 

where C- < 0 and (C+ > 0 are hxed parameters. The specihc choice of parameters (C+ 
does not play a substantial role, the most important is that they are of opposite sign, so 
that the lattice lies on both sides of the zero in M. 

The lattice £ comes from the Askey scheme of basic hypergeometric orthogonal poly¬ 
nomials (see Koekoek-Swarttouw [21|)- As explained in [S], the pre-limit zw-measures are 
closely related to certain unnamed finite systems of orthogonal polynomials on Z; in [8] , 
we called them the Askey-Lesky polynomials. It turns out that suitable g-analogs of the 
Askey-Lesky polynomials are the so-called pseudo big q-Jacobi polynomials (Koornwinder 
[27]i. These polynomials are eigenfunctions of a second order g-difference operator D (a 
natural g-analog of the difference operator associated with the Askey-Lesky polynomials) 
and they are orthogonal on £. 
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The idea to use the double lattice £ came to us when we became aware of the papers 
of Groenevelt and Koelink |19j . ra. m Thanks to them we realized that D admits a 
suitable selfadjoint version only when it is considered on a double lattice. 


1.2 Main results 


Very briefly, the main results of the present paper are the following: 

1. The description of the q-houndary of the extended Gelfand-Tsetlin graph. We show 
that the g-boundary can be identihed with the space of two-sided bounded countable 
point conhgurations on the lattice £. In the case of the ordinary Gelfand-Tsetlin graph, 
various approaches to the description of the (g-) boundary were devised in Okounkov- 
Olshanski [30], Borodin-Olshanski [9], Gorin Id. Petrov [33]. Our approach is different; 
it combines quantative estimates with some ideas from Gorin-Panova [TH] . 

2. The construction of a q-analog of the boundary zw-measures. These are certain 
probability measures on the g-boundary of the extended Gelfand-Tsetlin graph; they are 
obtained from the V-particle orthogonal polynomial ensembles corresponding to pseudo 
big g-Jacobi polynomials. The key point is the coherence property, the ensembles with 
different values of V = 1,2,... are linked to each other by means of certain canonical 
stochastic matrices associated with the graph. The coherency property is a nontrivial 
hypergeometric identitity; various combinatorial proofs of a similar identity in the case of 
the ordinary Gelfand-Tsetlin graph were suggested in Olshanski [32], but our approach 
is different: we show that our g-identity can be derived from the backward shift relation 
satished by the big g-Jacobi polynomials. 

3. The computation of the correlation functions. We prove that the measures in item 2 
above are determinantal measures and so are completely determined by their correlation 
kernels. Those have the form 


K{x,y) = const 


J^o{x)T'i{y) - T'i{x)J^o{y) 
x-y 


X,?/ e £, 


( 1 . 1 ) 


where To{x) and iFi{y) are certain functions on £ expressed through the basic hypergeo¬ 
metric function Kernels of such a form as in (1.1), are called integrable (in the sense 
of Its-Izergin-Korepin-Slavnov, see Deift’s survey paper [ID]). There are a lot of examples 
of integrable kernels coming from different models of random matrix theory and other 
sources, but the above kernel seems to be the hrst one which is expressed through basic 
hypergeometric functions. 


1.3 Organization of the paper 

In Section]^ we introduce the extended Gelfand-Tsetlin graph and the associated canon¬ 
ical stochastic matrices linking its levels; we call these matrices the q-links. 

Section is devoted to the g-boundary. We start with generalities concerning the 
notion of boundary that we need: it is dehned as the set of extreme points of a projective 
limit of simplices. Then we pass to our concrete situation where the simplices in question 
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are spanned by the vertices from the levels of the extended Gelfand-Tsetlin graph and 
the maps between the simplices are given by the g-links. The description of the boundary 
is given in Theorems 3^, |3.10[ 3.11 3.12[ Together they constitute our hrst main result. 
At the end of the section we define the correlation functions for arbitrary probability 
measures on the boundary and show that they can be obtained, in principle, by a large- 
limit transition. This abstract result is then used in the computation of Section 

Section deals with the pre-limit q-zw-measures — a g-analog of the pre-limit zw- 
measures. We start with their dehnition. Next we explain their connection with the 
pseudo big g-Jacobi polynomials and collect a number of formulas that we need (here 
our basic source is recent Koornwinder’s paper [27], which gives a further reference to 
Groenevelt-Koelink |2I])- Finally, we establish the coherency property of the pre-limit 


g-zw-measures (Theorem 4.7). It implies the existence of the boundary g-zw-measures, 
which is our second main result. 

In the hnal Section we compute the correlation kernels of the boundary g-zw- 
measures (Theorem 5.2). This is the third main result. The computation is a bit tedious 
because we have to manipulate with long formulas, but it is quite elementary: the desired 
kernel is obtained by a direct limit transition in the N-th Ghristoffel-Darboux kernel for 
the pseudo big g-Jacobi polynomials as —)• oo. 


1.4 Acknowledgment 
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Erik Koelink; their ideas helped us very much. We are also grateful to Erik Koelink for 
valuable comments. 
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2 The extended Gelfand-Tsetlin graph and q-links 
2.1 The extended Gelfand-Tsetlin graph G 

Here we introduce a novel object —- the extended Gelfand-Tsetlin graph and explain 
how it is related to the conventional Gelfand-Tsetlin graph. 

We recall the definition of the double g-lattice £ C M: 

£ ;= {(_q^ : m e Z} U {C+g*^ : n G Z} C M \ {0}, 

where g G (0,1), C- < 0, and C+ > 0 ^ire parameters. Unless otherwise stated, these three 
parameters are assumed to be fixed. 

By a configuration on £ we mean a subset X C £. Elements of X are called particles. 
If X is hnite, we enumerate its particles in the increasing order and write X = {xi < 
■ ■ ■ < xn)- The set of Y-particle conhgurations is denoted by Gn, N = 1,2,... . 
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We need a special notion of the interval I {a, h) between two points a < 6 of £: 

{ [a, 6) n £, a < b < 0, 

[a, 6] n -C, a < 0 < 6, 

(a, 6] U £, 0 < a < 6 

Note that if a < 0 < 6, then the interval I{a, b) contains inhnitely many points. 
We say that two conhgurations X G G^v+i and Y E Gn interlace if 

Hi e I{xi,Xi+i), i = l,...,N. 


Then we write y -< X or X >- F. 

Note that the interval I{xi,X]sf+i) is split into the disjoint union of N subintervals 
I{xi,Xi^i), i = 1, ... ,N. The interlacement condition Y -< X means that each of these 
subintervals contains precisely one particle from Y, see Figure for an example. 

I{xi,X2) I{x2,Xs) /(X3,X4) 

-€- )£ - ^^ - 

Xi X2 0 X3 X4 


Figure 1: Example of partition into intervals I{xi, Xj+i). If F -< X, then there is precisely 
one ^/-particle in each interval. 

Equivalently, the interlacement relation can be described in the following way. Let us 
modify the above dehnition of interval by setting 

{ (a, 6] n a < b < 0, 

(a, 6) n a < 0 < b, 

[a, 6) U £, 0 < a < b, 

where we also allow a = —00 and 6 = + 00 . Given F = {yi < ■ ■ ■ < yisf) we split the whole 
lattice £ into X + 1 intervals I{yi, yi+i), i = 0,..., N, where j/o := —00 and yw+i ■= + 00 . 
In this notation, X >- F means that the each such interval contains exactly one particle 
from X, see Figure]^ for an example. 

7(-cx),yi) 7 (?/i,?/2) 7(2/2, 2 / 3 ) 7(^3, cx)) 

- ^ - )£ - )£ - 

yi 0 y2 yz 


Figure 2: Example of partition into intervals /(2/i, 2/i+i)- If ^ Y X, then there is precisely 
one x-particle in each interval. 
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Definition 2.1. The extended Gelfand-Tsetlin graph G is the graded graph whose vertex 
set is the disjoint union Gi U G 2 U ... and the edges are formed by the pairs X Y oi 
interlacing configurations. We call Gat the Nth level of the graph. 

As an abstract graph, G does not depend on the three parameters g,C-,C+) but be¬ 
low we equip G with a supplementary structure which is defined with the use of these 
parameters. 

Recall that the ordinary Gelfand-Tsetlin graph (see e.g. 0,117] and references therein), 
is the graded graph GT whose iVth level GT^r [N = 1, 2,...) consists of signatures of 
length N: these are Wdimensional vectors v = {vi > ■ ■ ■ > vn) ^ ■ The edges of GT 

are formed by pairs v oi interlaced signatures, where X >~ u means 

Xi ^ I'i ^ i = 1,..., iV, A G GT N+ii ^ ^ GTat. 

There is a natural isomorphism between GT and the subgraph G"*" C G whose vertices 
are the configurations entirely contained in the positive part of the lattice £; here we 
use the notation 

= {C±q'" : n e Z} C £. 

The map GT —)■ G+ is given by 

GTn eG+, Xi-.= z = 1,..., W 

This justifies the name “extended Gelfand-Tsetlin graph” given to G. 


2.2 The g-links 

Here we introduce the g-links — stochastic matrices linking the levels of the graph G. 
These matrices arise when we equip the edges of the graph G with natural weights de¬ 
pending on parameter q. 


Definition 2.2. By the eotransition probabilities in G we mean the infinite sequence of 
matrices A^, A|,..., where is the matrix of format Gat+i x Gat with the entries 


Ar'(X,F) 


nf=i \y^ 
0 , 




Y\i<i<j<N^yj yd 


n 


l<i<j<N+l\^j 


Xi) 


Y ^X, 
otherwise. 


These matrices are called q-links. 

Remark 2.3. On the subgraph G"*", the g-links coincide with the matrices defined in 
Gorin HT). The g —?■ 1 limit of these matrices describes the branching rule for normalized 
irreducible characters of the groups t/(A^ -|- 1) under restriction onto U{N) G U{N + 1), 

cf. [SIIITIEI]. 
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Proposition 2.4. The q-links are stochastic matrices, i.e. for each N = 1,2,, all the 
matrix elements of are non-negative and for every X G Gat+i 


A~+‘(A',y) = L 

yeGjv 

For the proof we need a preparation. We endow every edge X >- Y with a weight 
defined by 

wt(X,y) :=|F| :=nbl 

y&y 

and we extend this definition to any pair of vertices from adjacent levels by setting 
wt(X, y) = 0 if X and Y do not interlace. 

Next, for any pair X E G^, Y ^ Gk, where N > K, we set 

Dim(X, Y) := ^ wt(X, X^^-^'>) wt{X^^-^\ ... wt{X^^+^\ Y) 

= |y| |x(^-')|...|x(^+')| 

XyX(^-'^)y-yX(^+^)yY 

{ii N — K = 1, then Dim(X, Y) := wt(X, X)). We call Dim(X, Y) the skew dimension. 
The dimensioT^ of X G Gat is defined in a similar way: 

DimX:= ^ wt(X, wt(X(^-^), ... wt(X(2), 

^ |X(^-b|...|x(i)| = ^ Dim(X,X«). 

X;-x(Jv-i)>-...>-x(i) x(i)eGi 

By convention, the dimension of every vertex of Gi is equal to 1. 

Lemma 2.5. For every a,b E 2, such that a < b one has 

6--a” = (l-g") n = l,2,.... 

c&I{a,b) 

Proof. We examine separately three cases: 0<a<6, a<6<0, and a < 0 < 6, and in 
all cases the claim is reduced to summation of geometric progressions. □ 

In our notations below we use the g-Pochhammer symbol: 

(a; g)n = (1 - a)(l - a ■ g) • • • (1 - a ■ 


Lemma 2.6. 

by 


The quantity DimX is finite for any N and any X E Gpf, and it is given 


DimX 




( 2 . 1 ) 


^For X C the limit quantity limg_>iDimX coincides with the dimension of the corresponding 
irreducible representation of the unitary group U{N). 




Proof. For = 1 the claim holds true: both sides of the formula equal 1. Then we use 
induction by N. Let us temporarily denote the right-hand side of (2.1) by Dim'X. By 
the very dehnition of the dimension function it suffices to check the recurrence relation 

Dim'X= |F|Dim'F. 

Y:Y^X 

Using the Vandermonde determinant evaluation, it reduces to the relation 




( 2 . 2 ) 


Y:Y^X 


We transform the matrix in the left-hand side of (2.2) by subtracting from the j + 1-th 
colnmn the j-th one, starting with j = N — 1 and ending with j = 1. The hrst row of the 
resulting matrix is (1, 0 ,..., 0) and the the determinant takes the form 


det [x* 


i+i 


X 


ihj=i- 


At this stage application of Lemma 2.5 completes the proof. 


□ 


Proof of Proposition jS.fl Comparing the dehnition of the g-link with the result of 

Lemma [2.61 we see that 


Ar^(x,v) = 


wt(X, V) DimV 


□ 


DimX 

On the other hand, by the very dehnition of the dimension fnnction, the matrix on the 
right is stochastic. 

For N > K we denote by A^ the matrix of format Gat x Gk with the entries 

Dim(A:,y) DimV 


A-(x,y) = 


Dim V 


From the dehnition of the dimension it is clear that A^ is a stochastic matrix and one 
has 

\N _ \N 

Xr — ■ 


2.3 Characterization of g-links 

In this section we explain how g-links are related to Schnr polynomials. We start with 
recalling a few basic formulas. 

Let Y denote the set of all partitions, which we identify with the Yonng diagrams. 
The length of a partition is the number of its nonzero parts (eqnivalently, the nnmber of 
nonzero rows of the corresponding Young diagram). We denote by Y{N) the subset of 
partitions with length at most N. It can be identihed with a subset of GTr (the set of 
signatures of length N) consisting of signatures with non-negative coordinates. Clearly, 
Y(l) C Y(2) C .... 
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We denote by S'aiat the iV-variate Schur polynomial indexed by a signature A G GT^v- 
It is a symmetric Laurent polynomial given by 


S\\n{uij ..., Un) — 




If A G Y(A^) C GTn, then S'aiat is an ordinary polynomial. 

The following formula is known as the q-specialization formula for S'aiat: 


^AiAT(i,...,g^ ^) = n 

1<2<_7<A^ 


qXj-\-N—j _ qXi^N—i 

qN-j _ qN-i 


(2.3) 


(see e.g. Macdonald [281 Ch. I, Section 3, Example 1]). It is immediately obtained from 
the Vandermonde determinant evaluation. 

Let us set 


= (AT - 1, V - 2,..., 0), = (q^i+^-\q^2+JV-2^... q^^) ^ 


The following symmetry relation is easily obtained from the dehnition of the Schur 
polynomials: 

—f —7-^ —7-A, i/GGTat. (2.4) 

Sx\Niq^^) S„iN{q^^) 

Given u G Y{N) and X = (xi < • • • < xn) G Gat, we set S'jy|Ar(X) = S'i,|at(xi, ... ,xn) 

S.In{X) S,|7v(X) 


and 


5hv(X) := 


(2.5) 


The next proposition provides a characterization of the g-links by a system of linear 
equations on their entries. 


Proposition 2.7. Let N > K and X G Gat be fixed. 

(i) The row entries A^{X,Y), where Y ranges over Gk, satisfy the following system 
of relations 

Y, A^(X, V)S,|^(V) = S,|at(X), z/ G Y(X). (2.6) 

YgGk 


(ii) Conversely, let Mk he a probability measure on Gk such that Mk{Y) vanishes 
unless Y is contained in a certain bounded interval [a,b] C M. If the quantities Mk{Y) 
satisfy the system (0 , that is, 


Y Mk{Y)S,,\k{Y) = ^,|7v(X), z/ G Y(X), (2.7) 

YgGk 


then Mk = A^(X, ■). 

Observe that for any K < N and Y G GTat, the quantity A^{X,Y) vanishes unless 
Y is contained in the closed interval [xi,XAr] C M. It follows that the probability measure 
A^{X, ■) on Ga: satishes the assumption of item (ii) above. 
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Proof, (i) Since ^ for — A' > 2, it suffices to prove (2.6) in the case 

A' = — 1. Then it takes the form 


Y:Y^X 






where we assume u G Y(A^ — 1), write X = {xi < ■ ■ ■ < xn), Y = {yi < ■ ■ ■ < Vn-i), 
and the factor (—1)^“^ on the right arises because the product ~ ^j) 

denominator of the formula for S^\n differs by the factor (—from the product 
Y\i^Axj — Xi) in our formula for Aj^_^. 


Using the formula for the g-specialization of the Schur polynomials (2.3) we may 
rewrite the desired equality as 


Af-l 


= (- 1 ) 




q 


i/i+N—i\ 


i=l 


Y:Y-<X 




Because u G Y(A^ ~ 1); we have = 0, so that the last row of the matrix in the 
left-hand side has the form (1,..., 1). Thus, we can argue as in the proof of Lemma 2.6 


we transform the matrix in the same way and then apply Lemma 2.5 

(ii) The set Gk can be viewed as the subset of formed by the vectors whose 
coordinates belong to £ and strictly increase. Therefore, there is a unique extension 
of Mk to a symmetric measure of total mass A"! on C M^, where “symmetric” 
means “invariant under permutations of the coordinates in . Let us denote the latter 
measure by Mk- By the assumption, the support of is bounded, so that Mk is 
uniquely determined by its moments. ^ 

On the other hand, the moments of Mk are uniquely determined by the relations 


(2.7). Indeed, these relations provide an explicit expression for the integrals of arbitrary 
Schur polynomials against Mk- Because Mk is symmetric and the Schur polynomials 


Sy\K span the whole space of symmetric polynomials in K variables, (2.7) contains the 
complete information about the moments of Mk- 

This completes the proof. □ 

We have already mentioned that on the subgraph G"*" C G, our g-links coincide with 
the g-links for the ordinary Gelfand-Tsetlin graph that were introduced in Gorin na. 
That is, assuming for simplicity = 1, we have 

= A^(A,p), A G GT^, p G GT^, 


where A^ is our notation for the g-links from na. 

As explained in na, the links A^ are characterized by the following system of relations: 
For hxed N > K and A G GTjv, one has 




A^ 


(A,/i) 


S^\k{ui^ . . . , Uk) 


Sx\Niui,...,UK,q ^,---,q^ 


yl-K) 


>S'a|v(1, q 


-1 


yl-N\ 


( 2 . 8 ) 
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Let us show how (2.8) is connected to (|^. We rewrite ( |2.8[ ) in the form 

UKq^~^) Sx\N{uiq^~^,UKq^~^, • • •, 1 ) 


Y, 


//GGTx 
N ow let us substitute 


Sfj,\K{q^ ^ , 1 ) 


Sx\N{q 


N-l 


, 1 ) 


= ueY{K), 


and next apply the symmetry relation (2.4). Then we get (2.6). 


Remark 2.8. The relations (2.8) say that the links come from the branching rule for 
appropriately normalized irreducible characters of the unitary groups. This suggests the 
idea to interpret the functions u i—)■ S'i.|Ar(X) as a surrogate of characters. 


3 The Q'-boundary of the extended Gelfand-Tsetlin 
graph 

The aim of this section is to identify the boundary of the extended Gelfand-Tsetlin graph 
equipped with g-links. For a historic motivation, we remark that the identihcation of 
the boundary of the conventional Gelfand-Tsetlin graph at g = 1 is equivalent to the 
classihcation of extreme characters of the group U{oo) (the extreme characters are in a 
one-to-one correspondence with the quasi-equivalence classes of hnite factor respresenta- 
tions), cf. [SI [31] and references therein. 

3.1 Preliminaries on boundaries 

In this short preliminary section we recall general dehnitions and results related to the 
notion of a “boundary”. The material is more or less standard, but various authors present 
it in different ways, see e.g. Dynkin [T3|, Diaconis-Freedman m, Kerov-Okounkov- 
Olshanski [23], Winkler [33] . 

Let A4{X) denote the set of all probability Borel measures on a given Borel (=mea- 
surable) space X. By a link between Borel spaces X and we mean a Markov kernel 
A = A{x,dy). We will need only the case when the second space is discrete. Then a 
link is a function A{x, y) on X x T), with nonnegative values, which is a Borel function in 
X E X, and such that 2/) = 1 for any x E X. In particular, if X is also discrete, 

this means that A is a stochastic matrix of format X x We represent a link by a dash 
arrow. Every link A : X —-> induces a map AA{X) —)■ AA{^), which is an affine map of 
convex sets. We write it as M i—>■ MA, where the measure M E AA{X) is viewed as a row 
vector. 

Assume we are given an inhnite chain 

• • • —> —> Gty —> ■ ■ ■ —> G2 —^ 
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of discrete spaces together with links : ^n+i ^n- These links give rise to maps 

Ad(ffAr+i) —)■ Ad(f2Ar), which allow us to form the projective limit space l^m Adfffjy). 
By the very dehnition of projective limit, elements of the space ^mAd(ffAr) are inhnite 
sequences {Mat G Ad(ffAr) : = 1,2,... } satisfying the coherency relation 

= iV = l,2,... . 

Such sequences {Mat} are called coherent systems (of probability measures). 

In what follows we assume that the set ^imAd(flAf) is nonempty. We equip it with the 
Borel structure inherited from the natural embedding 

OO 

N=l 

The product space on the right has a natural structure of a convex set and l^m Ad (flAr) 
is its convex subset. 

Definition 3.1. By the minimal boundary of such a chain {flAr, = 1, 2,... } we 

mean the set 12 := Ex(^imAl(12Ar)) of the extreme points of ^imAl(12Ar). Given a; G 12, 

we will denote by AfG) = {AfG^} the coherent system represented by oj. 

Below we often drop the adjective “minimal” and write simply “boundary”. 

Theorem 3.2. The set Q is a Borel subset of the projective limit space ^imAd(12Ar), so 
that we may form the space Ad (12) of probability Borel measures on 12. 

For every coherent system M = {AfAr} there exists a unique measure a G Ad (12) such 
that M = M^‘^^a(duj) in the sense that 

MN{y) = / M^\y)a{duj) for every A^ = 1, 2,... and every y G Gat. 

Jq 

Conversely, every measure a G Ad (12) generates in this way a coherent system, so that 
we get a bijection Ad (12) — Ijm Ad(12Af). 

Proof. See Olshanski [3ll Theorem 9.2]. □ 

We will say that a is the boundary measure of the corresponding coherent system 
M = {M^}. 

For every N we dehne a link A“ : 12 —->• Gat by setting A^{uj,y) = M^ff\y) for 
y E Qn- Here we use the fact that 12 is a Borel subset, which guarantees that A“(a;, y) is 
a Borel function in u. Note that 

AOO aA^+1 — aoo /y _ 1 9 

^i-Ar+l^'-V ~ ) iV — i, Z, . . . , 

where the product on the left is the natural composition. 
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Example 3.3 (The Pascal triangle). Here is a simple illustrative example. Set : = 
{0,1,..., N} and define the links by 

A N+^ / A \ A N+^ / A + 1 - m 


iv + r 


N + l 


with all other entries n) being equal to 0. Then the boundary Vl can be identified 

with the unit segment [0,1] and the links A“ have the form 


A“(a;,n)= (^)a;"(l-a;) 


\N-n 


These claims follow from the classical de Finetti theorem, cf. Feller [TTl ch. VII, Section 

4]. 

For any N > K we define the link A^ : Hjv as the composition 

A V _ A V 

1\k — ^^N-1 ■ ■ ■ ■ 

Definition 3.4. An infinite sequence {a:(iV) G Hat} is called regular if for fixed every 
K = 1,2,... there exists a weak limit of probability measures on VLk, 


Mr := lim A^{x{N), ■), 

N^oo 


which is a probability measure, too. (Since Qr is discrete, this simply means conver¬ 
gence on every subset of Hx).) Two regular sequences are said to be equivalent if the 
corresponding limit measures coincide for all K. The set of equivalence classes of regular 
sequences is called the Martin boundary of the chain (Hat, 

It is readily seen that the limit measures Mr, K = 1,2,... coming from a regular 
sequence form a coherent system. This allows us to identify the Martin boundary with a 
set of coherent systems and hence with a subset of Ad ( lim Hat). 

Theorem 3.5. As a subset of AA{I^AIn), the Martin boundary contains the minimal 
boundary H = Ex(Ad(^mr2Ar))- In other words, any point a; G H can be approximated by 
a sequence {x{N) G Hat} in the sense that 

lim A^{x{N),y) = A^{uj,y) 

N^oo 

for every fixed K and y G AIr . 

Proof. See Okounkov-Olshanski [301 Theorem 6.1]. □ 

It may happen that the Martin boundary is strictly larger than the minimal boundary. 
However, in many concrete examples both boundaries coincide. 

For instance, in the case of the Pascal triangle, the Martin and minimal boundaries 
coincide, and the convergence of a sequence n{N) to a boundary point t G [0,1] means 
that limAr->.oo''^(A^)/A^ = t- 
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3.2 The spaces Gqo and Gqo 

In this section we explain the basic properties of the set Goo, which will later turn out to 
be the desired boundary of the extended Gelfand-Tsetlin graph. 

We say that a conhguration X on ii is hounded if it is contained in a bounded closed 
segment [a,b] C M. The minimal such segment is denoted by I{X). Let Goo denote the 
set of all bounded conhgurations on £ and Goo C Goo be the set of all infinite bounded 
conhgurations. Obviously, 


Goo — Goo L-l {0} LI Gi U G2 U . . . . 


Let us enumerate elements of a configuration X G Goo in the decreasing order of their 
absolute values. To resolve a possible ambiguity arising when two elements of opposite 
sign have the same absolute value (which may happen only if log^ |C-| — loggC+ ^ we 
agree to put the positive element first. We denote by (a:(i), a;( 2 ),...) the resulting sequence 
and complete it with infinitely many O’s in the case when X hnite. 

Definition 3.6. We call (a;(i), a;( 2 ),...) the variational series of X. 

We set £ = £U{0} and equip this set with the topology induced from the ambient space 
M. Let be the infinite product space £ x £ x ... equipped with the product topology. 
The correspondence X 1 —>■ (a:(i),a;( 2 ),...) determines an injective map Goo —t £“, so that 
we may regard Goo as a subset of £°° and equip it with the induced topology. 

Proposition 3.7. In this topology, Goo is a locally compact space. 

Proof. From the definition of the variational series it follows that its terms decay expo¬ 
nentially fast. More precisely, one has 

k(2n)| < |a;(i)|g”"\ \x( 2 n+i)\ <\x[i)\q'", n = l,2,..., (3.1) 


because among the first m terms of the series one can always find at least [m/2] terms 
of the same sign. It follows from (3.1) that every subset of the form {X G Goo : |3^(i)| G 
const} is compact. On the other hand, such subsets are open and, as the constant factor 
gets large, they exhaust the whole space. This completes the proof. □ 


The next proposition shows that the topology on Goo can be also described without 


recourse to variational series. We use this result below in the proof of Corollary 3.18 


Proposition 3.8. A fundamental system of neighborhoods of a given configuration 
X G Goo is formed by the “e-neighborhoods” VfiX), e > 0, where V^^X) consists of 
all configurations that coincide with X outside the open interval 


Proof. We examine separately two cases: X G Goo and X G Gat for some N. 
Step 1. Assume X G Goo- For k = 1,2,... we set 


Uk{X) — {Z G Goo : ^(1) — ..., z^k) — X(k)}- 
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By the very definition of the topology in Goo, the sets Uk{X) form a fnndamental system 
of neighborhoods of X (here we use the fact that x^n) 7 ^ 0 for all n = 1,2,... . 

For every fixed k, the set 14(X) is contained in Uk{X) if £ is so small that all the 
points a;(i),..., X(^k) he outside {—e, e). 

On the other hand, every set 14 (X) is open. Indeed, let k be the first integer such 
that X(^k+i) lies in the interval (—e, e). Then all points X(^n) with n>k + l also he in that 
interval. It follows that 1^(X) coincides with set 

{Z G Goo : 2:(i) = a:(i),. . . , Z(k) = X(k), ^ 

which is obviously open. This proves the desired claim in the case X G Goo- 

Step 2. Assume now X G Gn- Then we modify the above argument as follows. We 
take k > N and replace the sets Uk{X) by the sets of the form 

Uk,s{X) = {Z e Goo ■ Z(^l) = X(^l),. .. ,z^N) = 4(n)|<4 N + 1 < u < k}, 

where 5 > 0. These sets form a fundamental system of neighborhoods of X. 

Evidently, V^{X) is contained in Uk^s if ^ is so small than e < S and X lies outside 

(-£,£). 

Finally, we have to prove that Ve{X) is open. It actually suffices to do this for small 
£ — so small that X lies outside {—e,e). Then 14(W) coincides with the open set 

{Z G Goo : Z[i) = a;(i),..., Z(^m) = X(n), 4(Ar+i)| < s}. 


This completes the proof. 


□ 


Note that a sequence {X(A^) G Gat : N = 1,2,...} converges to a configuration 
X G Goo if and only if the variational series of X (N) stabilizes to the variational series of 
X in the sense that for every fixed /c = 1, 2,..., there exists No{k) such that X(k){N) = X{k) 
for all N > Noik). 

Note also that any configuration X G Goo is a limit of a sequence {X(A^) E Gn ■ N = 
1, 2,... }. For instance, one can take as X{N) the collection of the first N terms of the 
variational series of X. 

We need one more remark. Let Si, denote the Schur function indexed by a partition 
n G Y. Given X G Goo, we denote by S'y(X) the evaluation of Si, at the collection 
{x : X E X}. The definition makes sense because, due to the bound (3.1), the sums 
^ fc = 1,2,. 


E 


x&X 


\x\ 


are finite. The same bound also shows that the function S,y{X) 


is continuous on 


3.3 Formulation of results 

For the sake of readability, our description of the boundary of the chain {Gat, is 

divided into four claims. 

Theorem 3.9. The Martin boundary of the chain {Gat, can be identified with the 

set Goo of infinite bounded configurations on the lattice 
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More precisely, a sequence {X(iV) G Gn} is regular if and only if {X{N)} stabilizes 
to a configuration X G Goo, and the correspondence {X(iV)} i—>• X established in this way 
determines a bijection of the Martin boundary onto Goo- 


Recall that every element of the Martin boundary is represented by a coherent system 
M = (Ml, M 2 ,...), and denote by M^"^^ ...) the coherent system corre¬ 

sponding to a given conhguration X G Goo- The next theorem provides a characterization 
of the measures . 

In the next theorem we use the notation 


Su{X) 


Su{X) 


cf. (2.5). 


Theorem 3.10. Let us fix X G Goo and K = 1,2,..., and let Y range over Gk- Then 
mI^\y) vanishes unless liY) C I{X) and we have 

Y, mY{X){Y)S,,\k{Y) = S,{X) for any z/ G Y{K). 

YeGK,I{Y)CI{X) 


(X) 

Moreover, ^ is the only probability distribution on Gk with these properties. 

The following theorem is a law of large numbers. 

Theorem 3.11. Let us fix X G Goo- Given L = 1,2,..., we consider Y G G^ as a 
random element with law Mf , so that the terms y(i){L),y{ 2 ){L ),... of the variational 
series ofY become random variables. 

For any fixed k = 1,2,..., the probability of the event y(k){L) = X(^k) tends to 1 as 
R —)■ cx). 


Finally, we identify the minimal boundary. 

Theorem 3.12. The minimal boundary of the chain {Gat, coincides with the Mar¬ 

tin boundary and hence can be identified with Goo- Moreover, under this identification the 
Borel structure of the minimal boundary coincides with the natural Borel structure of the 
space Goo- 

This implies, in particular, that the link has format Goo x G^: and, for every 
X G Goo, the probability distribution A^(X, ■) coincides with ■ 

The proofs are given in the next two subsections. 


3.4 Qualitative estimates 


A crucial part of the proof of theorems of Section 3^ is contained in the two qualitative 
lemmas which we present next. 
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Lemma 3.13. Fix an arbitrary k = 1,2,.... Assume N > k, let X ^ Gat be arbi¬ 
trary, and let Y G Gat-i be the random configuration distributed according to ■). 

Finally, let (x(i)) and {y{i)) be the variational series for X andY, respectively. 

Then for some c > 0 

e-cN 

Prob (?/(i) = a:(i ),... ,y^k) = x^k)) >1- 

where c might depend on q, (±, but not on X. 

Proof. Write as usual X = {xi < ■ ■ ■ < xat), Y = {yi < ■ ■ ■ < yN-i)- To make clear the 
main idea of the argument we hrst examine the simplest case when k = 1. There are 
two subcases depending on whether is positive or negative, but in these two cases the 
argument is the same. For dehniteness we will assume that a;(i) > 0. Then j/(i) = a;(i) 
means yN-i = xn, so that we have to prove that the probability of the event yN-i < xn 
is exponentially small as N gets large. 

Let us £x 1 / 1 ,..., yN -2 and consider the corresponding conditional distribution of yN-i- 
Below the symbol Prob( ■) refers to this distribution. 

It suffices to prove that Prob [yN-i < xn) is exponentially small with some constants 
that can be chosen independently of X and yi,..., yN- 2 - 

We introduce a shorthand notation: 


X := Xn, y := Vn-i, V ■= yN-2- 

We are going to prove the following two estimates (uniform on X and yi,..., yN-2)'- 


Prob {y = xq) < cic Prob {y = x), 

Prob (?/ < xq) < C 3 Prob {y = xq). 

They obviously imply the desired estimate. 

Recall that the probability of a given conhguration Y G Gat-i is given by 


(3.2) 

(3.3) 


N-l 


(g;g)Ar-in 1<2<_7<A^—1 iVj - Vi) 


n 




Xi) 


Ly-<x- 


= y 


N-2 

n 

2=1 


xq - Pi 
x-y,‘ 


(3.4) 


Prob(y)=Aj;_i(.Y,F)= nift 

2=1 

It implies 

Prob (y = xq) 

Prob [y = x) 

Evidently, all the N — 2 factors in the product on the right are strictly positive and less 
than 1. We claim that, moreover, they are uniformly separated from 1. Indeed, for every 
% = l,...,N-2, 

xq - yi _ q - yi/x 
x-yi I-yi/x' 

which does not exceed q if y^ is positive, and does not exceed (1 + g )/2 if y^ is negative 
(the latter holds because \yi\ <xn)- This implies (3.2). 
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To verify (3.3) we consider separately two cases depending on whether y* is positive 
or negative. 

Assume y* is positive, which means y* = xq^ with some positive integer m. Then, in 
take the values a 

Prob {jj = xg”) 


(3.3), y may take the values where n = 1, 2,..., m — 1, and we use the trivial bound 

N-2 


Prob [y = xq) 


= q 


n—1 


n 

i=l 


xq' - Vi ^ n-i 
— H 1 

xq - yi 


n = 1,2, 


(3.5) 


which immediately gives us ( |3.3 ). 

If y* is negative, then, in (3.3), y takes positive values xq,xq‘^,... and negative values 
y*q, y*q ‘^,.... The probabilities of the positive values are estimated as above. For the 
negative values we have 

Prob [y = y*q^) 


\y . 

/ X = - q 

Prob yy = xq) x 


\y^\ - \y*\f 


N-2 


n 

i=l 


\yi\ +xq 


< 9" ■ n 


\yi\ 


y- h\ +xq' 


(3.6) 


Since \yi\ < x for all z, every factor in the last product does not exceed (1 + q)~^. So the 
product becomes exponentially small as N gets large, while summation over n = 1, 2,... 
brings only the constant factor (1 — q)~^. This completes the proof of (3.3). 

Let us proceed now to the case k > 1. Let the hrst k — 1 terms of the variational 
series for X involve r negative elements and s positive elements, where r + s = k — 1, 
and assume for dehniteness that the kth term is positive (in the case it is negative, the 
argument is the same). We want to prove that, with exponentially small probability. 


z/i = Xj for z = 1,... , r; yN-j = Xn+i-j for j = 1, ..., s + 1. 

Using induction on k, it suffices to obtain a uniform exponentially small bound for the 
probability of the event y^^g-i < xn-s under the condition that the above equalities hold 
for all z = 1,..., r and for all j = 1,..., s, and all the remaining elements in Y are hxed. 

Now we can repeat the argument for the case k = 1. Below we only indicate necessary 
minor modihcations. Using the shorthand notation 


X xn-s, y yN-s-i, y* yN-s -2 


we reduce the desired claim to the same two estimates (3.2) and (3.3), as before. 
The equality (3.4) changes into 


Prob {y = xq) ^ ^ xq - yj yj - xq 

Prob {y = x) ^ fj- X - z/i yj - x ' 


(3.7) 


where Prob now denotes the conditional distribution of y^-s-i given the rest. The hrst 
product in (3.7) is still exponentially small as before. As for the second product, because 
x/yj < g, each of its factor admits the bound 


yj - xq ^ 1 - qx/yj ^ 1 

yj-x 1 - x/yj ~ I-q 
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so that that product does not exceed (1 — g) ^ 
Next, (3.5) changes into 


Prob {y = xq^) ^ ^-|4 " xq'^ - yj yj - xq^ 

Prob {y = xq) ^ | xq - y^ y^ - xq ’ 


Here the hrst product is at most 1, as before, while the second product is bounded from 
above by (1 — q^)~^■ 

Finally, (|3.6) takes the form 


Prob {y = y*q^) ^ M - \y*\(t Vj + b*!?' 

Prob {y = xq) x ^ {y^l + xq yj - xq 


Here we observe that, because \y*\ < yj, the second product is bounded from above by 
2^{l-q^)-\ 

These are all necessary modihcations. □ 


Given an A^-point conhguration X G Gn-, we denote by a;(i),... ,a;( 7 v) its variational 
series. We also denote by Y and j/(i),..., y{N-i) the random ■) distributed con¬ 

hguration in Gtv-i and its variational series, respectively. Since Y -< X with probability 1, 
we have \y{k)\ < k(fc)| for every k < N, which in turn implies that logq(||/(A;)|) > logg(|a:(fc)|). 


Lemma 3.14. Let fc = 1, 2,... be fixed. Then, as C > Q gets large, the probability of the 
event logg(||/(fc)|) — logg(|x(fc)|) > C tends to 0 uniformly on N > k and X E Gn- 


Proof. We closely follow the general plan of the proof of Lemma 3.13 and keep to the 
notation of that lemma. As in that lemma, we are dealing with conditional distributions. 
The difference is that instead of exponentially small bounds for large N we now establish 
uniform bounds for tails of distributions. 

We start again with the case k = 1 and we assume for dehniteness X(i) > 0. We have 
to hnd a uniform bound of the probability that the ratio \y\/x is small. The argument 
has two parts depending on the sign of y*. 

Assume y* is positive, i.e. y* = xg™, then y may take the values xg"' with n = 
0,... ,m — 1. Then we write (cf. (3.5)) 


Prob {y = xg") ^ xq^ - g, ^ 

Prob [y = x) JX ^ ~ Vi ~ 


n = 0,1,2,..., 


This means that the probability oi y/x = g” decays (as n — >■ -1-cxd) at least as fast 
as a geometric progression with the common ratio g. This gives a bound for the tail 
probabilities that does not depend on the length of the progression. 

Assume now y* is negative. Then y ranges over two inhnite sequences, the positive 
sequence {xg" : n = 0,1 ,... } and the negative sequence {y*q^ : n = 0,1 ,... }. 
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For the positive sequence, the tails are bounded exactly as above. For the negative 
sequence, the ratio \y\/x equals the product {\y*\lx)q^, and this quantity can be small 
due to either of the two factors, \y*\/x or g”. Now we modify (3.6) as follows 

N-2 


Prob (j/= \y*\ 

Prob (^y = x) x ^ 


N-2 

n 


\yi\ - ^ \y 


i=l 


This implies that 


\yi\ +x 

Prob(—xg” < j/ < O) 


X 


n 


\yi 


y- \yi\+x 


|y*l 

< —g". 

X 


< const ■ g"". 


Prob(g = x) 
which gives the desired uniform bound. 

The case fc > 2 is handled as in Lemma 3.13 we use recursion on k and slightly rehne 
the above arguments. □ 


3.5 Proofs of Theorems |3.9| , |3.10| , |3.11| , and |3.12 . 


Proof of Theorems \3.(^ and \3.10[ Step 1. Let {X(iV) e Gn : N = l,2,...}bea regular 
sequence and X(i)(A^),..., X(^n){N) denote the variational series for X{N). We claim that 
for every hxed fc, the sequence {x(fc)(iV)} stabilizes as N gets large. 

Indeed, let us £x L > 0 which is so large that 


n 


e 


c 


2 



where c is the constant from Lemma 3.13 for this k. Let 


be the random 


conhguration with law {X{N), ■) and be the probability measure on £ which serves 
as the distribution of the fcth term of the variational series of 

Due to the choice of L, Lemma 3.13 implies that gives weight > 2/3 to the point 
X(^k){X) G £, which in turn implies that the sum of the weights of all other points of £ is 
< 1/3. 

On the other hand, because {x(A^)} is regular, the measures A^{X{N), ■) weakly 
converge, as N ^ oo, to a probability measure on Gl, which entails that a similar claim 
holds for the probability measures they weakly converge to a probability measure on 
£. But this may only happen if the sequence {x(fc)(iV)} stabilizes. 

Evidently, the stable values of the terms X(^k){X) form a conhguration X G Gqo- 


Step 2. Conversely, let us suppose that a sequence {X(iV) G G^} stabilizes to a 
conhguration X G Goo- We claim that {X(A^) G Gtv} is regular. 

Indeed, let us hx k and prove that as N ^ oo the measures A^(X(iV), ■) on G^ 
weakly converge to a probability measure M^- 

To do this we first show that this sequence of measures is tight and then we prove 
that all limiting points of this sequence are the same. 

For that we hx any e: > 0 and choose L > 0 such that 


n 


e 


c 


> 1 — e, 
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where c is the constant from Lemma 3.13 for this k. Then this lemma implies that for 
large N, with probability greater than I — e, for the variational series 2 /( 2 ),... of 
{X{N), ■ )-random point conhgnration it holds that 

y(i)=X(i){N), y(^2) = X(2){N), y^k) = X(k){N), 

where, as above, {x(^i){N)) denotes the variational series for X{N). Now applying Lemma 
|3.14| and interlacing ineqnalities, we see that with probability greater than 1 — e the 
{X{N), ■ )-random point conhgnration stays bonnded away from zero and from inhnity. 
Since £ > 0 was arbitrary, this proves the tightness. 

Now let Mk be a probability measnre on Gk which is a weak limit for a snbseqnence 
of {A^ {X{N), ■). We claim that Mk is nniqnely determined by X. 


To see this we apply claim (i) of Proposition 2.7 It says that for any partition n G Y(fc) 
one has 

^ Aj'(V(Af),y)SH*(y) = S„|„(V(Af)). 

YeGk 

Let N go to inhnity along onr snbseqnence. Then the right-hand side tends to Siy{X) be- 
canse X{N) stabilizes at X. As for the left-hand side, it tends to XlveGfc ^k{y)Su\k{y)- 
here we combine the weak convergence {X{N), ■) —)■ Mk with the fact that the snp- 
ports of all the measnres from onr seqnence are nniformly bonnded. Therefore, we arrive 
to the system of relations 


Mk{Y)S,\kiY) = S,{X), u e Y{k), 


veG. 


and the proof of claim (ii) in Proposition 2.7 shows that this system determines Mk 
nniqnely. 


The argnments of these two steps prove Theorems 3.9 and 3.10 


□ 


Proof of Theorem 3.11. Let ns introdnce cylinder snbsets in Goo of the form 


^00 (^1) • • • ) Xk) {Z G Gqo • ^( 1 ) Xi, ... , ^{k) Xk }, 

where Xi,... ,Xk is a given hnite seqnence of points of £. For L > k, we denote by 
G 2 ,(xi, ..., Xk) the similar cylinder snbset in G^,. 

Next, let X G Goo be hxed and (x(i), a:( 2 ),...) be the corresponding variational 
series. Theorem |3.9[ that has been jnst proved, assigns to X a coh erent system 

can be 


3.11 


M^Y = : L = 1,2,...}. In these terms, the claim of Theorem 

rephrased in the following way: For every fixed k = 1,2,..., the mass given by Mf to 
the snbset Gl{x(^i'), ..., X(^k)) C. Gl tends to 1 as L —)• 00 . 

Let ns prove the last statement. Given an arbitrary e > 0 we choose Lq so large that 


nil 

n=Lo 


>1-G 
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where c is the constant from Lemma 3.13 that correspond to our hxed k. Next, let 
X{N ) G Gn stand for the conhguration formed by X(i),... ,X(^n)- By virtue of Lemma 
3.13, for any N > L > Lq, the mass assigned by the measure (X(iV), ■) to the subset 


GL(a;(i),... ,X(fc) is > 1 - 

On the other hand, Theorem 3.9 says us that our sequence {X(iV)} is regular and 
represents the coherent system so that, as iV —)■ oo, the measures A^{X{N), ■) 

converge to the measure M^\ It follows that > 1 — £ for any 

L > Lq. 

This completes the proof. □ 


Proof of Theorem 3.1S\ We will prove the claims of the theorem in the reverse order. 

Step 1. The correspondence X i—>• jg an injective map Goo —t ^mWl(GAr). We 
claim that the image of Goo is a Borel subset and the map is a Borel isomorphism onto 
this image. 

Indeed, observe that both spaces Goo and ^imAl(G 7 v) are obviously standard Borel 
spaces. Therefore, by virtue of an abstract theorem (see Mackey |29l Theorem 3.2]), it 
suffices to check that the map in question is Borel. By the dehnition of the Borel structure 
in ^imAl(GAr) this amounts to verifying that the functions on Goo of the form 

/x,y(W) :=Mf)(r), K=l,2,..., YeGk, 

are Borel. 

Now let X{N) G Gat be composed from the hrst N terms of the variational series of 
X. By Theorem |3.9| 

W’(y)= lim Ai(X(N),Y). 

N^oo 

It follows that fx^viX) is a pointwise limit of cylinder functions and hence is Borel. 

Step 2. Let X G Goo be arbitrary and let a be the boundary measure of which 


is a probability Borel measure on the minimal boundary (Theorem 3.2). We are going to 
prove that a is the delta-measure at X, which will imply that is extreme. 


By Theorem 3.5 the minimal boundary is contained in the Martin boundary, which 
can be identihed with Goo (Theorem 3.9). Next, by virtue of step 1, we may interpret a 
as an element of Wl(Goo)- Therefore, we may write 


and, consequently. 


Mf)= / Mf^>a{dZ), L = l,2,... . 


= / M^^^a{dZ) 

J Z^Goo 

dZ) 


We are going to show that a must be the delta-measure at X, which will imply that 
X is extreme. To this end it suffices to prove that for every A; = 1, 2,..., the measure a 
is concentrated on Goo(a^(i), ■ ■ • ,X(k)), where (xp)) is the variational series for X. 

The argument based on Lemma |3.13 and used in the proof of Theorem 3.11 shows 
that for any e > 0 there exists L = L{k, e) such that for any Z G Goo one has 

M^\Gl{z(^i), ..., Z(^k))) > 1 — 
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where 2 ^( 2 ), • • •) denotes the variational series for Z. 

Assume now that there exists a cylinder subset Goo(a^i, • • ■, C Goo which is distinct 
from Goo(a:(i),..., a;(fc)) and has a strictly positive a-mass: 

(T(Goo(a:i,... ,a;fc)) = 5 > 0. 


Then for L = L(A:, e) we have 





'zeG 00 


m|^^(Gl(xi, ..., Xk))(T{dZ) > 5(l-e). 


On the other hand, the hrst integral is equal to 

M^j^\Gl{xi, .. .,Xk)) < e, 


so that £ > 5(1 — e). Choosing e sufficiently small we get a contradiction, which completes 
the proof. □ 


3.6 Approximation of boundary measures 


Recall that the abstract Theorem |3.2| assigns to an arbitrary coherent system M = 
{Mtv} G At (Oat) a probability measure a on the minimal boundary O. It is tempting 
to regard this boundary measure as a limit of the sequence {Mat}. And indeed, in a 
number of concrete cases this informal statement can be turned into a formal one (see 
e.g. Olshanski Theorem 10.2]. Here we show how to do this in our situation. 

In the theorem below M = {Mat} G hm Al(GAr) is an arbitrary coherent system and a 


is its boundary measure. By virtue of Theorem |3. 12| we may regard a as a probability Borel 
measure on Goo- Therefore, we may put all the probability measures under considerations, 
that is. Mat’s and a, on a common space — the space Goo introduced in Subsection 3^ 
Recall that it is a locally compact topological space. 


Theorem 3.15. Let {Mat} and a he as above. Then Mjq a in the weak topology of 
the space Ad (Goo)- 

Corollary 3.16. In particular, for every X G Goo, the measures Mfj = A“(X, •) weakly 
converge to the delta measure at X. 


Proof of Theorem 3.15 . 
We have to prove that 


Let C'(Goo) be the space of bounded continuous functions on Goo- 


lim (/,M 7 v) = (/,a), 

N^oo 


f e C(Goo), 


where the angular brackets denote the pairing between functions and measures. 
We have 

[ (/.AfJfVK’f)- 

J Goo 
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By Lebesgue’s dominated convergence theorem, it suffices to prove that 


N^oo 


X e 


/ e ^((Goo). 


We will derive this claim (which is just the claim of Cor ollary 3.16) from the law of large 


3.11 


numbers for the measures established in Theorem 

We use the notation of that theorem. Observe that there exists a compact set in Gqc 
containing X and the supports of all measures : for instance, one can take the set 


{Z G Goo : |^(i)| < |2^(i)|}- 


On this set, the function f{Z) can be approximated, in the uniform norm, by cylinder 
functions which are polynomials in hnitely many variables Z(^i^, Z(^ 2 ), ■ ■ ■ ■ Therefore we 
may assume that / itself is such a polynomial. But then the desired result follows from 
Theorem 13.111 □ 

Definition 3.17. Let a be an arbitrary Borel probability measure on Goo- The n-particle 
correlation function of a, denoted by p"(xi,... ,x„), assigns to an arbitrary u-tuple of 
pairwise different points xi,... ,Xn G £ the probability of the event that the cr-random 
conhguration contains all these points. 

The next corollary will be used in Section 

Corollary 3.18. Let {Mjy} be a coherent system and a be its boundary measure. For 
every fixed n = 1,2,... and any pairwise different Xi,..., G £, 


lim Pm^{xi, ...,Xn)= p'fixi,.. .,Xn). 

N^oo ™ 

Proof. Consider the function f{Z) on Goo which equals 1 if Z contains all the points 
xi,..., Xm and 0 otherwise. We have 


= {f,MN), p'^ixi, . . . , Xn) = {f,a). 


On the other hand, / is locally constant and hence continuous, as is immediately seen 
from Proposition |3.8[ Then the desired result follows from Theorem 3.15 □ 


4 The g-zw-measures 

4.1 Definition of the measures 

We are going to introduce a family of probability measures on the levels Gat of the 
extended Gelfand-Tsetlin graph G. This family is the main object of study in the present 
paper. The measures of the family depend on a quadruple (a, fi, 7 , 5) of parameters, 
which are complex or real numbers subject to some conditions. We hrst give the formula 
for the measures and then discuss the constraints on the parameters. 
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In what follows we use the conventional notation from g-analysis (see Gasper-Rahman 

[ 16 ]) 

oo / \ n—1 

(a; 9)=o := 11(1" = 0.1.2 ,.... 

m=0 m=0 

For = 1, 2,... and X = (xi < • • • < xn) G Gn we dehne 


N 




JV V , I, ^ l<i<j<N 


(4.1) 


where 


w 




(x) = |a;| 


(ax, (3x; q)^ 


N (7gi-^a;,5gi-^a;;g)oo’ 

and ZN(a, f3,'y,6) is a normalization constant, 


X E Z, 


N 

ZN{a,/3,-f,6) = ^ JJw^’^’^’^(a:i) JJ (xj - Xif 

X&Gn i=l ^<i<j<N 

(a closed expression for Z^ia, /3, 7 , 5) is given in Section [4^ below). Of course, for X = 1 
the product over i < j is missing. 

The conditions on the parameters (a, (3, 7 , 5) must guarantee the correctness of the 
dehnition. That is, w°^^''^'^(x) has to be a nonnegative real number for all N and all 
X E Z, and ^Ar(a, (3, 7 , 5) has to be hnite for all N. 

There are two cases, the nondegenerate and degenerate ones. In the nodegenerate 
case the quantity w'^^'^'^(x) is always strictly positive, and in the degenerate case it may 
vanish for some x E Z. 

We are mostly interested in the nondegenerate case, which is described with the help 
of the following lemma. 

Lemma 4.1. Assume (a,b) is a pair of nonzero complex numbers such that one of the 
following two conditions holds: 

(i) either a and b are nonreal and complex-conjugate 

(ii) or a and b are real and such that a~^ and b~^ are contained in an open interval 
between two neighboring points of the lattice Z. 

Then the quantity (ax,bx;q)oo is real and strictly positive for all x E Z. 

Proof. Condition (i) guarantees that (ax; g)oo and ( 6 x; q)oo are nonzero and complex- 
conjugate. Condition (ii) guarantees that these two quantities are real, nonzero, and of 
the same sign. □ 


One can easily prove the converse statement: if a and b are nonzero and such that 
(ax, bx] q)oo is real and strictly positive for every x E Z, then either (i) or (ii) holds true. 

It is evident that any of the conditions (i), (ii) implies that ab is real and strictly 
positive. One more evident observation: if (a,b) satishles (i) or (ii), then the same holds 
true for (ag”, bq"^) with n eTj. 
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Definition 4.2. We say that a quadruple of parameters is admissible and 

nondegenerate if both pairs (a, and ( 7 ,5) satisfy the assumptions on the previous 
lemma and, moreover, 'jdq > afd. 


Proposition 4.3. If {a,/3,'y,d) is admissible and nondegenerate in the sense of the previ¬ 
ous definition, then the definition (4.1) of the probability measures makes sense. 


Proof. The conditions on the parameters guarantee that > 0 for every N and 

all X E 2, so the question to settle is whether the sum, which we denoted by ft, 7 , 5), 

is finite. We will prove that 


|a:|* < 00 , i = 0,1,... ,2N - 2, N = l,2,..., (4.2) 

x£S. 


from which the desired conclusion Z^i^a, ft,'j,6) < 00 evidently follows. 


Note that Ixl 


X 


—)■ 1 as X G £ approaches 0 , which implies the convergence 
of the part of the sum ( 4.2[ ) corresponding to all x in a finite neighborhood of the origin. 

Let us examine the asymptotics of {x) as x goes to plus or minus infinity along 

£. It is convenient to introduce the shorthand notation 


{a,h,c,d) = {a, ft,'yq^ ^ ,5q^ ^), w{x) 


(ax, bx; g)oo 
(ex, dx] g)oo 


Every point x G £ has the form x = C,±q where n E 1^. In this notation, x goes to 
infinity means that n goes to + 00 . Then we have 


w(C,±q ”) 


«±,K±-^q)oo (g ^C±^)n f ay\"" ^ / ab \ 

(cC±,dC±;q)oo (c-iCi\d-iC±^g)n \cdqj \cdqj 


(4.3) 


It follows that 


—n\ 2—2N j. / 

(C±g )X ~ const I — 


n —)■ +CX). 


This shows that needed sum is indeed finite. 


□ 


The degenerate case splits into several subcases. Here is one of them. 

Example 4.4. Assume a and fl are real, a < 0 < ft, and a~^ E £, G £. As for 7 
and 5, assume that they are nonreal and complex-conjugate. Then (x) is strictly 

positive if a~^q < x < ft~^q, and vanishes otherwise. In this case, the measures are also 
correctly defined. 


Unless otherwise stated, in what follows we will assume that (a,ft,'y,S) is admissible 
and nondegenerate in the sense of Definition |4.2[ However, our results are automatically 
extended to various variants of the degenerate cases. In particular, one may choose the 
parameters as in Example |4.4 above. 
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4.2 Pseudo big q-Jacobi polynomials 


The definition (4.1) of the measure on Ga? fits into the general scheme of or¬ 


thogonal polynomial ensembles, see, e.g., Konig’s survey paper [25]. In our situation, 
the orthogonal polynomials in question are those associated with the weight function 
on £. These polynomials are not mentioned in the encyclopedic paper by 
Koekoek-Swarttouw [23]. Fortunately, they appeared in recent Koornwinder’s paper 
which is an addendum to m)- Below we collect the necessary information from m- 


Definition 4.5. The pseudo big g-Jacobi polynomials with parameters {a,b,c,d) are de¬ 
fined by 


Pn{x) = Pn{x; a, b, c,d) := 3^2 


fq cda ^b 

y cb~^q, ca~^q 



n = 0,1,2 ,..., 


where we use the conventional notation 


302 


4 , B, C 
D, E 


Z : = 


q)n{C-, q)n ^r. 
^ iD;q)n{E;q)n{q;q)n 


The key property of the polynomials Pn{a,b,c,d) is that they are orthogonal on £ 
with weight 

{ax;q)oo{bx;q)o, 


w{x) = w{x-, a, b, c, d) = |x| 


X e £. 


(cx;g)oo(dx;g)oc 

Here we tacitly assume that the parameters are such that the weight is nonnegative. 
We are mainly interested in the nondegenerate case, when the weight is nowhere vanish¬ 
ing. Then, as is seen from (4.3), it possesses only finitely many moments, so that the 
corresponding family of orthogonal polynomials is finite, too. The maximal degree of 
orthogonal polynomials is the largest integer n such that ^ 

It is worth noting that the polynomials Pn{x', a, b, c, d) do not depend on the extra pa¬ 
rameters C-) C-I-) which enter the definition of the lattice £, the support of the measure. So, 
fixing (a, b, c, d) we still dispose of a two-parameter family of different orthogonality mea¬ 
sures. Of course, the non-uniqueness of the orthogonality measure is readily explained, 
because the number of orthogonal polynomials is finite. However, the possibility to explic¬ 
itly exhibit a whole family of orthogonality measures is a remarkable fact. It also shows 
that the role of parameters {(-, y+) is less important than that of parameters (a, b, c, d). 

The polynomials Pn{x) are closely related to the classical big q-Jacobi polynomials, cf. 
Andrews-Askey [1], Koekoek-Swarttouw [23], Ismail [22], Koornwinder [26], defined via 


Vn{x; A, B, C) — 302 


Aq, Cq 


X 


Namely, let us assume that the two sets of parameters are related by 

A-l, S=-. 

baa 


(4.4) 









Then the comparison of the formulas above shows that 


Pn{x; a, b, c, d) = Vn{cx] A, B, C). 


(4.5) 


Looking at (4.5) it might seem that our polynomials merely coincide with classical Vn 
up to a rescaling of the variable. This is not quite true, as the admissible domains of 
parameters {a,b,c,d and A, B, C) for these two families of polynomials are different and 
the supports of their orthogonality measures are also different. A more precise point 
of view is that the polynomials Pn and Vn are analytic continuations (in parameters) of 
each other. Nevertheless, relation (4.5) makes it possible to extract a number of necessary 
formulas for the polynomials Pn from well-known formulas for the big g-Jacobi polynomials 
V 

Note that the above expression for Pn{x] a, b, c, d) is evidently symmetric under trans¬ 
position a b, and there is also a hidden symmetry under transposition c ■H- d: namely, 
the monic polynomials (which are obtained when we divide by its top degree coeffi¬ 
cient) are symmetric under c ■H- d. 

Let hn{a, b, c, d) denote the squared norm of P„(x; a, b, c, d): 

hn{a, b, c, d) = '^^{Pn{x; a, b, c, d))‘^w{x; a, b, c, d). 

Here is an explicit expression for this quantity (see Koornwinder m Section 14.5] and 
references therein): 


hn{a, b, c, d) 
ho{a, b, c, d) 




o2 \ ^ 


^n{n-l)/2^2n 


{q,qd/a,qd/b;q), 


1 — qcd/(ab) 


{qcd/ (a5), gc/a, qc/b; q)n 1 — cd/{ab) 

(4.6) 


and 


ho{a,b,c,d) = C+ 


(g, a/c, a/d, b/c, b/d; g)oo 0g(C-/C+, cdC-C+) 


(4.7) 


{ab/{qcd);q)oc ^^(cC-, dC-, <+, c(C+) ’ 

where the theta function 9q{u) of a single argument u is defined by 

9g{u) = {u,q/x;q)^ 
and we use the shorthand notation 

("Ui, . . . , Umi q)n (^li d')n ■ ■ ■ (^mj d'jni 9q(^Uiy . . . , Urn) ■ dg('Ui) ■ ■ ■ 9 q{Urn^ ■ 

We also need the top degree coefficient of Pn{x;a,b,c,d), which we denote by 
kn{a, b, c, d). It is readily obtained from the dehnition of the polynomial: 


/c„(a, b, c, d) = d 
Proposition 4.6. In this notation we have 

Af-l 


(cdg"'+^/(a5); q)j 
{cq/b,cq/a]q)n 


(4.8) 


ZN{a,/3,'y,6) = JJ 


hn{a,ld,iq , dg^ ^) 
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Proof. This is a general fact, which holds for any orthogonal polynomial ensemble (see 
e.g. the computation in Konig [25l Section 2.7]). □ 

It is a remarkable fact that there exists a closed expression for the sum 

ho(a, b, c, d) := a, b, c, d). 

This is a true g-analog of famous Dougall’s formula [12] for the bilateral hypergeometric 
series 2 H 2 sX 1. It seems that no closed expression exists for the sum of the weights 
w{x] a, b, c, d) over the positive part of £, which would correspond to a naive extension of 
Dougall’s formula (see Askey [ 2 | and Bailey |3|). 


4.3 Coherency via orthogonal polynomials. 


The aim of this section is to prove the following theorem. 

Theorem 4.7. Assume that the quadru ple (a, d, 7 , 5) of parameters is admissible and 
nondegenerate in the sense of Definition f.2. Then the measures on N = 

1 , 2 ,. 


given by (4.1) form a coherent system on the graph 


Before proceeding to the proof we need to perform some preparatory work. In the 
argument below we apply a trick consisting in introducing some “virtual particles”. We 
learnt it from Alexei Borodin; see Borodin-Ferrari-Prahofer-Sasamoto 0 Lemma 3.4]. 

If X, ?/ G £, we write x <\ y ii either y is positive and x < y oi ii y negative and x < y. 
Next, introduce a kernel on £ x (£ U {+ 00 }) by setting 


A{x,y) = 


1, X <i y 
0 , otherwise 


with the understanding that A{x, +00) = 1 for every x G £. 

Lemma 4.8. Let X = (xi < ■ • • < xat+i) G Gn+i, ^ = ivi < 
Vn+i ■= +00. 

Then 

det[A(xi,2/j)]fi\ = 1 ^’ , 

U, otherwise 


< i/v) ^ GjV; and 


Proof. As explained in Subsection |2.1[ the points yo := — 00 , yi, ..., j/^r, j/v+i = +00 
divide £ into a disjoint union into N + 1 intervals /(j/j_i, j/j), i = 1,..., X + 1, with the 
agreement that the Ah interval includes its right endpoint yi if and only if |/j < 0, and 
includes its left endpoint j/i_i if and only if 0 < yi-i. 

If two of the points xi,..., xat+i belong to the same interval, then the corresponding 
two rows of the matrix A{xi,yj) are the same and the determinant vanishes. Thus, for 
non-zero determinant, all the points xi,..., xn+i need to belong to different intervals, so 
that Xi G Ji,..., Xat+i G In+i- A case-by-case check shows that this just means X >- Y. 

Further, in this case, the matrix A{x,y) is triangular with Is on the diagonal and, 
thus, its determinant is one. □ 
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Lemma 4.9. Let us set 


w{x) = w{x; a, b, c, d), Pn{x) = Pn{x; a, b, c, d), 
w*{x) = w{x-,a,b,cq~^,dq~^), P*^i{x) = Pn+i{x-,a,b,cq~^,dq~^] 

where n > 0 is such that ^ Then for every G £ one has 


xGS,: x<y 


cq 


{b — c) {a — c) 


w{y)Pn{y) 

\y\ ■ 


(4.9) 


Proof. Observe that as y ^ —oo the left-hand side of (4.9) tends to 0. The right-hand 
side also tends to zero dne to the condition on n. Similarly, in the limit y —?• +oo, the 
left-hand side of (4.9) becomes the scalar prodnct of P^j^i with constant fnnction and 
hence vanishes and so is the right-hand side. 

Therefore, it snffices to check that the difference of (4.9) at ?/ G £ and at yq is a valid 
identity. 

If y is negative, then x < y means x < y and the difference on the left is 

-w*{yq)P*+i{yq). 

If y is positive, then x <\y means x < y and the difference on the left is 


w*{yq)P*^^{yq). 

Therefore, in both cases the desired identity can be written as 

yw*{yq)Pf+^{yq) = {w{y)Pn{y) - q~^w{yq)Pn{yq)) 

or eqnivalently 


yP*+i{yq) = 


cq 


{b — c){a — c) \w*{yq) 




w*{yq)' 


We claim that the latter relation follows from the backward shift relation for the big 
g-Jacobi polynomials (Koekoek and Swarttonw [211 (3.5.8)]) 


{1-A){1-C)uVn+i{u;Aq ^,Bq ^,Cq ^) 

= {u-A){u- C)Vn{u] A B, C) - A(u - l)(Bu - C)Pn(uq; A, B, C). 

Indeed, let ns set u = cy and compare the two relations using the connection between 
the two families of polynomials. 

We have 


P*+^{yq) = Pn+i{yq,a,b,cq \dq ^) = Vn+i{cy; Aq \Bq \Cq ^). 
This shows that the left hand sides of our relations differ by a scalar factor. 
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Let us examine now the right-hand sides. We have 


Pn{y) = Vniu] A B, C), Pn{yq) = Vn{uq] A, B, C) 


and 


w{y) = \y\ 


{ay, by-q)^^ 

{cy,dy]q)A 
It follows that 

w{y) 


w{yq) = q\y\ 


{ayq, byq; g)oo 

{cyq,dyq;q)A 


w*{yq) = q\y\ 


{ayq, byq; q)c 

{cy,dy,q)oc 


w*{yq) 

On the other hand, 


= q {^-ay){^-by), q 


-1 w{yq) 
w*{yq) 


= q {^-cy){l-dy). 


{u - A){u -C) = —(1 - ay){l - hy), 


A{u - l){Bu -C) = —(1 - cy){l - dy), 


which shows that the right-hand sides of our relations differ by the same constant factor 
as above. 

This completes the proof. □ 


Proof of Theorem fA. We prove that 

A V+l _ kja,l3,'y,5 


iV= 1,2,... . 


Since and are probability measures, it suffices to prove a formally weaker 

claim that the both sides differ by a nonzero constant factor. This will substantially 
simplify the computations, because we can ignore the cumbersome expression for the 
normalization factor entering the dehnition of our measures. On the other hand, if one 
reconstructs all the normalization factors in the present proof, then this gives an inde¬ 
pendent check of the explicit formula for Z]si{a, j3, 7 , 5). 

Throughout the proof we denote by “const” a (possibly varying) nonzero constant 


factor whose exact value is not relevant for us. We keep to the notation of Lemma 4.9 
where we set 

(a,6,c,d) = (a,/?,7g^"^, V^)- 

Taking into account the dehnition of the measures and the links, the desired relation can 
be written as 

= const det[||/jr^u;(%)l/^^-^]f;i,=i, VF e Gn 

X:XyY 

or equivalently as 

det[w*{xi)Pf_^{xi)]Y2i = const det[\yf-A{yj)Pk-i{yj)]YY VF G Gjv- 

X:XyY 
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Next, applying Lemma [4.8| we may write the left-hand side as 

det [w* det [A{xi, . 

xi<---<xm+i 

Applying the Cauchy-Binet formula and taking into account the dehnition of the kernel 
A{x, y) we write the above sum as a single determinant 

det[B{j,k)]f+}^, B{j,k):= w*{x)P^_^{x). 

X-. x<yj 

Now we use the fact that i/n+i = +C) 0 . It follows that 


xdS, 


Because of the orthogonality of the polynomials this expression vanishes unless fc = 1, 
when it is a positive constant. Therefore, our determinant of order -|- 1 is reduced, up 
to a scalar factor, to the minor of order N which is obtained by removing the {N + l)th 
row and the hrst column. This minor has the form 


det 


w*{x)Pl{x) 

X'. x<3yj 


N 


j,k=l 


Finally we apply Lemma 4.9 It says that this determinant is equal to 


constdet [\yj\ ^w{yj)Pk-i{yj)]Yi ^ 


which is precisely what we want. 


□ 


Corollary 4.10. Assume that the quadruple 


nondegenerate in the sense of Definition \4.2 
the coherent system 


3.2 


[a, (3,^,6) of parameters is admissible and 
Then, by virtue of the abstr act Theorem 


•■N 


N = 1,2,...} afforded by Theorem f.l gives rise to 


a probability measure on the space Gc 


Definition 4.11. The probability measure from the above corollary will be denoted by 
and called a boundary q-zw-measure. 


5 Correlation functions of the boundary q-zw- 
measures 


Our dehnition of the boundary g-zw-measures (Dehnition 4.11 above) is non¬ 

constructive in the sense that it relies on an abstract existence theorem. So the question 
arises whether it is possible to obtain a more concrete description of these measures. There 
is no hope that they can be given by a density with respect to some evident reference 
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measure like Lebesgue measure on the line. This is a standard situation for measures on 
inhnite-dimensional spaces. In the case of the ordinary Gelfand-Tsetlin graph, the bound¬ 
ary zw-measures were described in Borodin-Olshanski [ 8 ] via their correlation functions. 
We follow the same approach and hnd in this section the correlation functions of 

We recall that the dehnition of correlation functions adapted to our concrete situation 


was given in Dehnition 3.17 


The correlation functions of are computed in terms of certain basic hyperge¬ 

ometric functions that we now introduce: 




= \ X 


where 


{xa, xf3] q) 
6q{x^)eq{x5) 


oo i_r 
X 


q2r-2.q 


201 


0 r-1 

-Q 

7 ox 


a(3 

7(5 

A, B 
C 


■ 2 <; 


/ aq'^ ^ 
\ 


6 ’ /5x 

6x 


^ 1 ^ 


7 


r G Z, (5.1) 


/ 


^ \ ^ (A; q)n{B] q)n ^r. 

' ^ (C';g)n(g;g)n 


Remark 5.1. The function is invariant under the transpositions o; 77 /5 and 

7 77 5. This can be verihed using transformation formulas for the 20 i series, see Gasper- 
Rahman [T^ (III.3) and (II.2)]: 


f)r = C 


A,B 

C 


Further, for r G Z we set 

{i^y 


^ I ^ {ABZ/C-qU ^^ 


q 


2 -H 


(■^j *?)oo 

{C/B,BZ-q) 

{C,Z;q)^ 


9, I 7 


(C/A, C/B 
C 




Gbz/c.b 


BZ 


ABZ/C 
C/B 


a r 


-1 «^r-l Pr-l y 


-q , -rq 

7 0 


,r-l 


7 


- 1 ^<2(7C-, <^C-, 7C+, <^C+) 


ajS 


,2r-2 


ajS 


7(5 ^ ’7(5 


q2r 2. q 


(5.2) 


Theorem 5.2. Take any admissible quadruple a, (3,^,6 such that afd < q'^'yS. Then the 
correlation functions p^, n = 1, 2 ,... of AA/Aa3 computed via 

p'^ixi, ...,Xn) = det[K°‘’‘^’'^’\xi,Xj)]lj^-^, 

y) = — . Mx)Aiiy) - Ai{x)Ao{y) 

f)i x-y 

where the singularity at x = y in the last formula should be resolved using the L’Hospital’s 
rule. 
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Remark 5.3. It might seem that the dehnition of through and (5.1) 

makes sense only when /3 < q'j, since otherwise the series dehning 2 <Pi diverges. However, 
a/3 < guarantees that either |a| < q\'y\, or |/3| < q\'y\, or |a| < q\6\, or \(3\ < q\6\. 
Since does no change when we swap a 7-)- /3 and 7 -f-)- 5 we can then extend the 
dehnition to all 4 cases. In fact, we believe (but leave this out of the scope of the present 


paper) that Theorem 5.2 should hold for any admissible quadruple of parameters (that 


is, if instead of a/3 < q^'yS we impose the weaker requirement a(3 < q'yS as in Dehnition 


4.2) provided that we replace by its analytic continuation in parameters 

a,/ 3 , 7 ,( 5 . 


The rest of this section is devoted to the proof of Theorem 5^ We start by computing 
correlation functions of the measures . 

Proposition 5.4. Take any admissible quadruple a,/ 3 , 7 , 5 . Let w{x) = 
w{x]a,/3,'jq^~^,6q^~^), Pn{x) = Pn{x;a, ,6q^~^) and let hn, kn he the squared 


norms and leading coefficients of these polynomials, as defined in Section \472 . 
Then the correlation functions p'fi, n = 1,2,... of are computed via 




(5.3) 


T^a,/3,7,<5/ N ^V-1 r~rT — ^NiffiPN-liy) - PN-l{x)PNiy) /f. .N 

Kn {x,y) = -— - ^,/w{x)w{y) -, 5.4 

kw hN-i x-y 

where the singularity at x = y in the last formula should be resolved using the L’HospitaTs 
rule. 

Proof. The admissibility condition implies that the hrst N orthogonal polynomials 
Fq, ■ • • ,Pn-i are well-dehned and belong to £ 2 ( 12 , w). Then a general theorem (see e.g. 
Borodin [4], Borodin-Gorin [3 Section 3]) for the A^-point ensembles on the real line with 
probability distribution of the form 

1 ^ 

Prob(a;i,... ,a;Ar) = — ]^M;(a;i) {xi - xjf 


2 = 1 


1<2<J'<A^ 


known as orthogonal polynomial ensemble with weight w{x) implies that the correlation 


functions are of the form (5.3) with 


Kf’-'px.y) = ,/w(x)w(y) ■ 


Pn{x)Pn(y) 

^ h 

22=0 


n 


Applying the Christohel-Darboux formula (see e.g. Szego [SU Section 3.2]) we rewrite 
Kn^''^’^{ x,y) as (5.4). We do not have to worry if the polynomial Pn{x) is square inte- 
grable, because the Christoffel-Darboux formula relies only on the three-term relations 
xPn = ... for n = 0,..., iV — 1, which make sense. 
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In formula (5.4), the indeterminacy on the diagonal x = y is resolved via the 
L’Hospital’s rule. Alternatively, we may use the Cauchy integral and write 


Kn{x,x) = 7^ / KNix,y) (5.5) 

zm J y — X 

with integration over a small simple ^/-contour around x. □ 

The correlation functions for can be obtained by the limit transition of Corol¬ 
lary |3.18| and for that we need to send —)■ oo in all the parts of the formula for 

y). We rely on the following statement. 

Lemma 5.5. For any fixed parameters B^C^D^E such that \D\ > q, D ^ , E ^ q^, 

we have as integer n tends to -|-oo 


3g>2 


q '^,Bq ^,Cq 
Dq~'^, Eq~^ 


BC 

Ide 


q-n(n-l)/2 


\BC'^) PC 


D D 

~2g>i 1 

BC 


(5.6) 

Proof. ( |5.6 ) is proved by applying the transformation formula in Gasper-Rahman [ISl 
(III. 11)], which reads 


302 


q ^^Bq ‘^,Cq 
Dq~'^, Eq~"‘ 


^ , M-.il (¥) «-” . 


^—n ^ ^ 
^ ^ B’’ C 

Da-n Dp 

^q ) BC 


(5.7) 


Next, to handle {Eq ”■)„, we use 




(5.8) 


It remains to notice that as n — >■ -I-cxd, each term in the series expansion of 302 in the 
right-hand side of (5.7) degenerates to the same term in the expansion for 20i indicated 
in (5.6). Since the series expansion of 302 is majorated by a multiple of geometric series 
with ratio \q/D\ < 1, this gives the desired convergence of 302 to 20i and proves (5.6). □ 

As a corollary we find the asymptotics of the polynomials Pn- 


Proposition 5.6. Take an admissible quadruple 0,0,7,5 and any r G Z such that 
PN-r{x) is a well-defined orthogonal polynomial for all large N, and such that \q'^~^(3\ < 
| 7 |. We have 


lim (sgnx)^ ^ 

N^OO 


(7^)(V-1)/2 

qN{N-l)/2 


\/w{x] a, 0, 7 g^"^, 5q^-^) 


PN-rjx] a, 0 , 'yq^~^, 6q^~^) 
kN-r{<y, 13, 7 ?^"^, dq^-^) 

= Pr{x). 


(5.9) 


The convergence in (5.9) holds for any a: G £. Moreover, for each xq G £ the con¬ 
vergence in ( |5.9 ) is uniform over x in an open complex neighborhood of xq if we replace 
everywhere sgn(x) by a/x^/x with branch of square root chosen so that \/x^/xo = sgn(xo), 


and similarly replace |x| by vx^ in all the definitions 
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Proof. We start by investigating the asymptotic of ■\/w{x). We have 
w{x-, a, /3,= ta(a;; a, /3,7, 5) 


X 


(1 — ^xq ~^)... (1 — 7 a:g^“^)(l — 5xq~^) ... (1 — 6xq^~^) 

Ar(Ar-l) 2(V-1) 

= w{x;a,/3,'y,6)^ 


Note that 7 ^ > 0 while x may be negative, so that we write 

y/x^iN-i) = (xsgna:)^"^. 


Therefore, as iV —)■ 00 


(sgnx)^ ^■\/w{x; a, /3, 'yq^~^, 6q^~^) 


q 


N(N-l)/2 


{xa, xf3] q)c 


X 


(7(5)(-'v-i)/22,Af-i y i“'i 0^(^x'y,x6) 

We further compute the large-iV asymptotics of PN-r{x]a,/3,'yq^ ^,6q^ ^). By the 


dehnition of Section 4.2 


PN-r{,X]a,l3,'yq^ ^) = 302 


„-Af+r J^^3-N-r 
y 5 a/?" ’ 

7 2-V ^ 2—w 

p a 


(5.10) 


Now we use (5.6) with parameters 


n = N-r, B = C = -fxq^-\ D = ^q^-\ E = -q^-\ 

ap pa 

This gives 

Pn-v{x- a, 0, 7g^-^, 5q^-^) ~ ■ q-{N-r){N-r-l)/2-r{N-r) 

((f~^a q 


X 


SL 

6x 


a 


-q 


r-1 


7 


■ 2 P 1 


6 ’ /3x 

V Sx 


q^ 

7 






For the leading coefficients kn we use the formula (4.8) and transformation (5.8) which 
yield 

(jll^-2r+3 r-N. \ 

kN-.(a, D, W-P SP-P = 


('7„2—r^r—V 7 „2—r^r—Af. 

yj^q q 1 aq q ’ qjN-r 

2r-2. N 

Combining the asymptotic of ■\/w(x), Pjy^rix), and /cAr.r we arrive at the desired result. 

□ 
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Proposition 5.7. As N ^ oo 


/iAr_r(a,/3,7g^ 

i-i ■ 


Remark 5.8. In fact, under appropriate restrictions on the quadruple a, (3,^,6 and r 
the functions sxe orthogonal in with counting measure and f)r are their squared 
norms. This is merely the iV —)■ cx) limit of the orthogonality relations for polynomials 

R,. 


Proof of Proposition 5/7. By the formulas of Section |4.2[ we have 

hn{a, (3, 7g^“^, C+, C-) 


[/c„(a,/3,7gi-^,hgi-^)]2 


— c+ 


^ N—i ® N—1 N—1 N—1 

q, -q^^ \ -q^^ ^ ^ -q^^ q 

'y 0 'y 0 


2N-3 . 9/3. „ 

y 76 ’ y 


X 


9, i 799"-“C-C+ 


X (-1)’ 


^g(7y^"^C-, <5y^"^C-, 7y^"^C+, <5y^"^C+) 

^2n+n(n—1)/2 2W7^ — ^ 


{a/3y 


afS 

g3-2N+2nl3. _ I 
^ a/3 


7g2 N ^q2 N ^^2 N ^^2 N 


y;y 


a ’ /3 ’ a ’ (3 

3-2N+njl_ ^3-2N+nj3_ ri3-2N 1±- 


ap 


, y 


a/3 


, y" 


a/3 


; y 


(5.11) 


Set n = N — r and send —)■ cxd. Then using (5.8) and the identity 


Oq{qx) = {qx-,q)oc{x ] q)oo = [x] q)^{qx ^;g) 


-1 ^ 1 - ^ -la ( \ 

1 — X 


we see that asymptotically (5.11) behaves as 


/S 


C+^ 


Oq ( 7hC-C+ 


X 


7y 


a 


l-\-n—N ’ ^f^l+n—N ’ 


0,(7C-, 5C-, 7C+, ^C+) 

a (3 


■y 


2n+n(n—1)/2 _y 


-27V 


y q/3 


(7(5)” n3-2N+2njl _ 

' ' ' V Q,^ 


7y" 


^gl+n-Af’ ^gl+n-Af 


; y 


-2+2Ar-2na? ^-2+2N-2n9ll ri-2+2N-n9l3- 


7(3 


, y 


7^ 


, y 


7^ 


; y 


n^f (4(3-27V+n)-2(5-4Ar+3n)-(5-4V+n)) 


7V(V-1)^_^_ 
^ r\Ar_r 


C+^ 


^9 ( —> 7^C-C+ 


7^ 

a/3 


^ (7C-, 9C-. 7C+. «C+) 


X 


a 


/5 


(U 


/S 


7^1-r’ ^gl-r’ ^gl-r’ ^^1-r 




; y 


h □ (5.12) 
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Proof of Theorem 5-4 By Proposition 5.4 and Corollary 3.18 we need to study N ^ oo 
asymptotic of Note that both coherent systems and the kernel 

is invariant under swaps a (3 and 7 -f-)- 5. Therefore, we can assume 
by making an appropriate swap that \(3\ < glyl, since a(3 < guarantees that either 
|/3| < q\-f\ or |a| < q\6\. 

We now consider the case x ^ y. Then we use Propositions ^ and 5^ to get the 
required asymptotics and notice that the factors (sgnx)^“^(sgn cancel out when 

we compute the determinants in the dehnition of . 

We extend the asymptotic to the case x = y hj using the Cauchy integral (since 
before the limit we deal with polynomials, everything is analytic) and then passing in it 
to —)■ 00 limit by using the uniformity in Proposition 5.6 □ 
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